14 research outputs found
Simple Analysis of Sparse, Sign-Consistent JL
Allen-Zhu, Gelashvili, Micali, and Shavit construct a sparse, sign-consistent Johnson-Lindenstrauss distribution, and prove that this distribution yields an essentially optimal dimension for the correct choice of sparsity. However, their analysis of the upper bound on the dimension and sparsity requires a complicated combinatorial graph-based argument similar to Kane and Nelson\u27s analysis of sparse JL. We present a simple, combinatorics-free analysis of sparse, sign-consistent JL that yields the same dimension and sparsity upper bounds as the original analysis. Our analysis also yields dimension/sparsity tradeoffs, which were not previously known.
As with previous proofs in this area, our analysis is based on applying Markov\u27s inequality to the pth moment of an error term that can be expressed as a quadratic form of Rademacher variables. Interestingly, we show that, unlike in previous work in the area, the traditionally used Hanson-Wright bound is not strong enough to yield our desired result. Indeed, although the Hanson-Wright bound is known to be optimal for gaussian degree-2 chaos, it was already shown to be suboptimal for Rademachers. Surprisingly, we are able to show a simple moment bound for quadratic forms of Rademachers that is sufficiently tight to achieve our desired result, which given the ubiquity of moment and tail bounds in theoretical computer science, is likely to be of broader interest
Individual Fairness in Pipelines
It is well understood that a system built from individually fair components
may not itself be individually fair. In this work, we investigate individual
fairness under pipeline composition. Pipelines differ from ordinary sequential
or repeated composition in that individuals may drop out at any stage, and
classification in subsequent stages may depend on the remaining "cohort" of
individuals. As an example, a company might hire a team for a new project and
at a later point promote the highest performer on the team. Unlike other
repeated classification settings, where the degree of unfairness degrades
gracefully over multiple fair steps, the degree of unfairness in pipelines can
be arbitrary, even in a pipeline with just two stages.
Guided by a panoply of real-world examples, we provide a rigorous framework
for evaluating different types of fairness guarantees for pipelines. We show
that na\"{i}ve auditing is unable to uncover systematic unfairness and that, in
order to ensure fairness, some form of dependence must exist between the design
of algorithms at different stages in the pipeline. Finally, we provide
constructions that permit flexibility at later stages, meaning that there is no
need to lock in the entire pipeline at the time that the early stage is
constructed
Individual Fairness in Advertising Auctions Through Inverse Proportionality
Recent empirical work demonstrates that online advertisement can exhibit bias in the delivery of ads across users even when all advertisers bid in a non-discriminatory manner. We study the design ad auctions that, given fair bids, are guaranteed to produce fair outcomes. Following the works of Dwork and Ilvento [2019] and Chawla et al. [2020], our goal is to design a truthful auction that satisfies "individual fairness" in its outcomes: informally speaking, users that are similar to each other should obtain similar allocations of ads. Within this framework we quantify the tradeoff between social welfare maximization and fairness.
This work makes two conceptual contributions. First, we express the fairness constraint as a kind of stability condition: any two users that are assigned multiplicatively similar values by all the advertisers must receive additively similar allocations for each advertiser. This value stability constraint is expressed as a function that maps the multiplicative distance between value vectors to the maximum allowable ?_{?} distance between the corresponding allocations. Standard auctions do not satisfy this kind of value stability.
Second, we introduce a new class of allocation algorithms called Inverse Proportional Allocation that achieve a near optimal tradeoff between fairness and social welfare for a broad and expressive class of value stability conditions. These allocation algorithms are truthful and prior-free, and achieve a constant factor approximation to the optimal (unconstrained) social welfare. In particular, the approximation ratio is independent of the number of advertisers in the system. In this respect, these allocation algorithms greatly surpass the guarantees achieved in previous work. We also extend our results to broader notions of fairness that we call subset fairness
Supply-Side Equilibria in Recommender Systems
Algorithmic recommender systems such as Spotify and Netflix affect not only
consumer behavior but also producer incentives. Producers seek to create
content that will be shown by the recommendation algorithm, which can impact
both the diversity and quality of their content. In this work, we investigate
the resulting supply-side equilibria in personalized content recommender
systems. We model users and content as -dimensional vectors, the
recommendation algorithm as showing each user the content with highest dot
product, and producers as maximizing the number of users who are recommended
their content minus the cost of production. Two key features of our model are
that the producer decision space is multi-dimensional and the user base is
heterogeneous, which contrasts with classical low-dimensional models.
Multi-dimensionality and heterogeneity create the potential for
specialization, where different producers create different types of content at
equilibrium. Using a duality argument, we derive necessary and sufficient
conditions for whether specialization occurs: these conditions depend on the
extent to which users are heterogeneous and to which producers can perform well
on all dimensions at once without incurring a high cost. Then, we characterize
the distribution of content at equilibrium in concrete settings with two
populations of users. Lastly, we show that specialization can enable producers
to achieve positive profit at equilibrium, which means that specialization can
reduce the competitiveness of the marketplace. At a conceptual level, our
analysis of supply-side competition takes a step towards elucidating how
personalized recommendations shape the marketplace of digital goods, and
towards understanding what new phenomena arise in multi-dimensional competitive
settings.Comment: Updated version with revised and expanded conten
Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm
We study the function space characterization of the inductive bias resulting
from controlling the norm of the weights in linear convolutional
networks. We view this in terms of an induced regularizer in the function space
given by the minimum norm of weights required to realize a linear function. For
two layer linear convolutional networks with output channels and kernel
size , we show the following: (a) If the inputs to the network have a single
channel, the induced regularizer for any is a norm given by a semidefinite
program (SDP) that is independent of the number of output channels . We
further validate these results through a binary classification task on MNIST.
(b) In contrast, for networks with multi-channel inputs, multiple output
channels can be necessary to merely realize all matrix-valued linear functions
and thus the inductive bias does depend on . Further, for sufficiently large
, the induced regularizer for and are the nuclear norm and the
group-sparse norm, respectively, of the Fourier coefficients --
both of which promote sparse structures